The present invention relates to processing of video signals, and more particularly to a histogram-based segmentation of images and video via color moments.
In the processing of images or video signals it is desirable to be able to take an object, such as a tennis player, from one video signal or image and superimpose it upon another video signal or image. To this end keying systems were developedxe2x80x94either luminance or chrominance based. For example, in character generation luminance keying is typically used, while chrominance keying is used for placing a weather man in front of a weather map. Chrominance keying is based upon the object to be segmented, i.e., the weather man, being situated before a uniform color background, such as a blue screen. A key signal is generated that is one value when the color is blue and another value when the color is not blue. The key signal is then used to cut a hole in another video signal into which the segmented object is placed, thus superimposing an object from one video signal onto another.
In naturally occurring scenes there may be many objects against a non-uniform color background, such as tennis players and the ball against a crowd background. It may be desirable to segment an object from this scene in order to superimpose it upon another scene. In this situation conventional luminance and chrominance key generation techniques do not work.
Also in the proposed MPEG-7 standard it is desirable to be able to segment objects from an image so that objects may be separately compressed.
What is desired is a method of segmenting images and video using the colors of the objects within the images.
Accordingly the present invention provides histogram-based segmentation of images and video via color moments. A user defines a relatively large area that lies entirely within each object of interest in one or more images, frames or pictures from a video signal. A normalized, average color moment vector is generated with an associated co-variance matrix for each user-defined area, as well as xe2x80x9cgarbagexe2x80x9d parameters based upon the normalized average color moment and associated co-variance matrix. Each normalized average color moment vector defines a color class. A segmentation algorithm then examines each block of each image, frame or picture, deriving a color moment vector for each block. A log likelihood test is used to determine for each block of the image, frame or picture which color class does the block most likely fall into. Then a pair of xe2x80x9cgarbagexe2x80x9d model tests based on the xe2x80x9cgarbagexe2x80x9d parameters are conducted to assure that the block is within the most likely color class. If the block fails one of the xe2x80x9cgarbagexe2x80x9d model tests, then the block is classified as being a member of a xe2x80x9cgarbagexe2x80x9d color class. All connected blocks that fall within a given color class are determined about the centroid of the corresponding user-defined rectangle and are associated with the corresponding object for which a segmentation key is generated for the object. The centroid is tracked from frame to frame of the video using a variation of a Kalman filter.
The objects, advantages and other novel features of the present invention are apparent from the following detailed description when read in conjunction with the appended claims and attached drawing.